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ABSTRACT 

Intended for administrators and policy makers as well 
as teachers, this digest offers selection strategies for consumers of 
English/language arts computer software. Following the introduction, 
the digest discusses initial considerations when evaluating software, 
including examining both the accompanying documentation and the 
actual program. It chen outlines what to look for when first viewing 
the program in terms of organization and structure and the quality of 
the program's feedback for the user's answers. Next, the digest 
covers the pedagogical issues to consider during the first viewing. 
It then offers criteria for a second viewing, in which the evaluator 
intentionally makes errors that a student might make to examine how 
the program responds to confused or slow learners. Finally, the 
digest offers suggestions for dealing with software publishers. 
( HTH ) 
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SOFTWARE EVALUATION FOR THE TEACHER OF THE ENGLISH LANGUAGE ARTS 

by 

Pam Janello 

Sifting good software from bad is not difficult, but it is not like 
selecting textbooks. If you use the same strategies., you may waste 
both time and money. You can flip through many textbooks and get a 
good idea of their worth because you are sophisticated about what you 
are looking for. Also, when teaching with textbooks, you are still a 
major part of the show — preparing, emphasizing, explaining, 
highlighting, filling in the gaps, making up for weaknesses. The book 
will not slam shut if a student forgets to turn the pages in a certain 
manner. A computer program, however, can create frustration and 
confusion for the student. It can, in effect, "break down" if it is 
not thoughtfully designed. Therefore, teachers must learn selection 
strategies in order to become sophisticated consumers. 

What are the initial considerations? 

Never buy software without Lliorough examination. You should look at 
two parts of the software: the documentation and the actual, running 
program. The documentation is the written explanation that comes with 
the computer disk. Beware of any software that does not. have such 
documentation, as lack of documentation usually means the person who 
produced the program didn't think it through or didn't bother with 
explanatory material. 

The documentation sliould help answer some basic questions. Are the 
stated objectives of the program something you need for your 
students? What equipment specifications does the program take? If the 
program needs 64,000 units of memory and your computers have only 
48,000, do you want to buy additional memory chips for all your 
machines so that this program will run? Does the success of the 
program depend on color — if so, do yoa have color monitors? Trying to 
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distinguish the sentences written in red from the ones in blue on a 
black and white screen is tiresome, if not impossible. Does the 
program need disk drives or cassette players? If disk drives, how 
many? Is one of the strengths of the program that it will print out a 
paper copy of the student's work? If it is and you either don't have 
access to a printer or have one printer for thirty students, maybe the 
program isn't for you. 

When first viewing the program, what should you look for concerning 
organization and structure? 

Suppose you like the subject matter, accept the stated objectives, and 
have the equipment. It's time to look at the program for the first 
run-through — and this ought to be the first of several if you are 
going to be a competent consumer. Concentrate on three areas during 
the first run: Does the program carry out its stated objectives? Is 
it user-friendly? Does it have sound pedagogical structure? 

You probably do not need help deciding if the program carri^es out its 
stated objectives, because that's a regular part of teaching. -But what 
about user friendliness? Is the program eas>^ to use? Are there clear 
directions, or do you need technical knowledge about computers to make 
this program work? 

Examine the program for ease of operation, if you have to press the 
Return key after your input, does it tell you to do it? Is the program 
menu driven? That is, do you start out with a page of choices, or do 
you have to start in the same place every time-^-reading directions at 
the beginning? What about a way to quit if you get bogged down or must 
, leave? Can you get out of the program short of pulling the plug? Can 
you go back to see those directions you thought you didn't need in the 
first place, or thought that you'd remember after a quick read 
through? 

What should I know about feedback ? 

ERIC rj 
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It's important to look ^it both positive and negative feedback. These 
terms do not mean good and bad. Rather, they refer to the feedback 
given for correct and incorrect responses. Is the feedback 
appropriate? Too much positive feedback isn't useful. Some programs go 
too far, using the student's name and a string of superlatives to 

reward a correct response. "What a marvelous job, Charlie. You sure 
know your stuff." This approach is boring and often condescending. 
Field studies show that a simple OKAY or CORRECT is often enough. 

"Does the negative feedback help point students in the right direction? 
Computers can handle many possible answers in many forms in a helpful 
way — if the programming was done by a competent programmer and a 
designer who knows the content . ^Feedback that says "THINK!" to 
students who thought they were thinking but missed the answer anyway 
is useless. A more helpful feedback design is the corrective feedback 
paradigm (CFP) . This is a system that reruns missed questions through 
a questioning sequence at specified intervals to reinforce' retention. / 

Finally, there are some programs that won't let students out until 
they have answered five or ten or fifteen questions correctly in a row. 
Some students just can't do that, and they get so frustrated by a lack 
of success that they quit. Keep this in mind as you evaluate a 
program. In addition, remember to check whether the program keeps 
track of the student's work, giving some type of summary and 
evaluation at the end. 

What pedagogical issues shoilld_I__co nsider duri n g the f i rst viewing? 

There are several important issues. For example/ is the sequencing 
pedagogically logical? Does the program porrnit the student to 
inceract-- think, respond, wonder, predict--or does the student just 
read an electronic workbook? (You can buy quite a few workbooks for 
the cost of one piece of software, much less the computer.) The more 
interactive the program is the better. Get students involved in th:s 
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evaluation and make sure that involvement is worthwhile for both their 
education and their time. 

Look carefully at the language. Is it too formal, or too laden with 
slang? Is it so dependent on fad that it will be outdated in two 
years? Finally, do graphics and/or sound add anything to the' program, 
or are they window dressing? 

What should I look for in a second viewing? 

If the program seems to pass the test so far — you like t^e objectives, 
it's user friendly, and the pedagogy is good — it's time for a second 
round of assessment, in which you consciously make errors. This may 
seem hard. Teachers like to do things correctly and to be neat. On 
this run-through, however, you'll need to force yourself to be a 

confused and/or slow learner. 

' ■ ■• 

Find out what happens if you just hit the return key with no input. 
'Can you do it three times, get the correct answer, and page on through 
or do you have to make an attempt to answer? What does your program do" 
•fith perceptually correct but literally incorrect answar3--Gspecially 
lisspellings or parts of names, abbreviations? What does the program 
do with a totally off-the-wall response — treat it as such or say, 
"Wow, that's interesting, Johnnie." What happens if you hit the escape 
key, hit many keys simultaneously, or randomly tap the keyboard? Try 
it. If the program is capable of handling such things, it's said to 
be "bullet proof." if the program isn't designed to handle accidental 
or intentional problems, it's going to backfire on some students. 

If you look at these aspects, you will have given the program a good 
preview. Jf you still like the program at this point, then it's time 
to turn the program over to some of your students. Choose at least a 
couple — quick/bright and slow/unmotivated and get their opinions. 
Their actions and reactions will reveal much more about the programs. 
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What should I know about dealing with software publishers? 

If publishers will not let you inspect programs on a trial basis, 
beware. They are not entirely at fault, however. Software development, 
especially well-done material, costs money, if they send out a disk 
for preview and you copy it and send it back, they stand to lose a 
great deal of money. So allow for the publisher's qualms and make 
arrangements for previewing. Send a letter guaranteeing that you will 
not copy il^legally. In addition, request to see the real program / 
rather than shortened preview disks. If they won't send just the / 
disks, ask them to send a sales representative to do a demonstration /• 
and then try to get time for your own evaluation. Sometimes several / 
schools can' join together and invite several publishers^ to put on a 
software fair with a wider range of materials. It takes planning, but 
it is a way to see the material. 

Another factor to check on is how to get a backup disk or multiple 
copies. You should be given either a backup disk or provision to mak6 
your own. Also, how does the publisher deal with your need for five, 
ten, fifteen or thirty copies for your district? Paying the full 
retail price for each one could wipe out a budget for software on one 
item. Negotiate a reasonable, nominal fee for additional or 
replacement copies. In school settings disks are sometimes lost, 
stepped on, or otherwise damaged. In addition, you should investigate 
the publisher's policy if, for some reason you didn't foresee, the * 
program just will not work out. Do not, however, make the return 
option an excuse for weak evaluation practice on your part. 

Finally, deal openly and honest^.y with publishers and expect to be 
treated in the same manner. Remember, they are somewhat new at this 
also. The copyright laws are ambiguous on computer materials and 
business policies and practices are just being formed. Publishers and 
educators must work together to establish a smooth working 
relationship. Tell your colleagues and educators in other districts 

o 
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about publishers- who are good to work with. Set up a network of 
information. 

Make use of general professional resources, like reviews of software 
from EPIE (Educational Products Informational Services) and MicroSift, 
Read software reviews and ideas about instructional use of computer 
journals like Language Arts , English Jour nal, arid Computers in Reading 
and Language Arts . But don't let others' ideas take the place of your 
own hands-on assessment. The ideas in this digest, along with 
documents like "Guidelines for. Review and Evaluation of English 
Language Arts Software/ (National Council of Teachers of English^ 
1984) can help you to choose the best software for use with your 
^students. 
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. Sifting good software from bad is not difficult, but it is not 
like selecting textbooks. If you use the same strategies, you 
may waste both time and rroney. You can flip through many 
textbooks and get a good idea of their worth because you are 
sophisticated about what you are looking for. Also, when 
teaching with textbooks, you are still a major part of the 
show-preparing, emphasizing, explaining, highlighting, filling 
in the gaps, and making up for weaknesses. The book will 
not slam shut if a student forgers to turn the pages in a certain 
manner. A computer program, however, can create frustration 
and confusion for the student. It can, in effect, "^"eak down" 
if it is not thoughtfully designed. Therefore, teachers must 
learn selection strategies in order ko become sophisticated 
consumers. 

What Are the Initial Considerations? 

Never buy software without conducting a thorough examina- 
tion of it. You should look at two parts of the software: the 
documentation and the actual, running program. The docu- 
mentation is the written explanaticyn that comes with the 
computer disk. Beware of any software that does not have 
such documentation, as lack of documentation often means 
the person who produced the program didn't organize it well 
or didn't bother with explanatory material. 

The documentation should help answer some basic ques* 
tions. Are the stated objectives of the program something you 
need for your students? What equipment specifications does 
the program take? If the program needs 64,000 units of 
memory and your computers have only 48,000, do you want 
to buy additional memory chips for all your machines so that 
this proa'i^'i^lTi will run? Does the success of the program depend 
. on colo[-i^ so, 00 you have color monitors? Trying to dis- 
* tinguish the sentences written in red from the ones in blue 
on a black and white screen is tiresome, if not impossible. 
Does the program need disk drives or cassette players? If 
disk drives, how many? Is one of the strengths of the program 
,that it wiH print out a paper copy of the student's work? If it 
is and you either don't have access to a printer or have one 
printer for thirty students, maybe the program isn't for you. 

When First Viewing the Program, 
What Should You Look for 
Concerning Organization and Structure? 

Suppose you like the subject matter, accept the stated ob- 
jectives, and have the equipment. It's time to examine the 
^ orogram for the first run-through-and this ought to be the 

ERIC 



first of several if you are going to be a competent consumer. 
Concentrate on three areas during the first run: Does the 
program carry out its stated objectives? Is it user*friendly? 
Does it have sound pedagogical structure? 

You probably do not need help in deciding if the program 
carries out its stated- objectives, because that's a regular part 
of teaching. But what about user friendliness? Is the program 
easy to use? Are there clear directions, or do you need techni- 
cal knowledge about computers to make this program work? 

In addition, examine the program for ease of operation. 
If you have to press the return key after your input, does it 
tell you to do so? Is the program menu driven? That is, do you 
start out with a page of choices, or do you have to start in 
the same place every time-reading directions at the beginning? 
What about a way to quit if you get bogged down or must 
leave? Can you get out of the program sKr^«r of pulling the 
plug? Can you go back to see those directions you thought 
you didn't need in the first place, or thought that you'd 
remember after a quick read*through? 



What Should I Know about Feedback? 

It's important to look at both positive and negative feedback. 
These terms do not mean good and bad. Rather, they refer to 
the feedback given for correct and incorrect responses. Is the 
feedback appropriate? Too much positive feedback isn't 
useful. Some programs go too far, using the student's name 
and a string of superlatives to reward a correct response: 
"What a marvelous job, Charlie. You sure know your stuff." 
This approach is boring and often condescending. Field studies 
show that a simple "Okay" or "Correct" is often enouyh. 

Moreover, does the negative feedback help point students in 
the right direction? Computers can handle many possible 
answers in many forms in a helpful way-if the programming 
was done by a competent programmer and a designer who 
knows the content. Feedback that says "Think!" is useless 
to students who thought they were thinking but missed the 
answer anyway. A more helpful feedback design is thecprrec* 
tive feedback paradigm (CFP). This system reruns missed 
questions through a questioning sequence at specified intervals 
to reinforce retention. 

Finally, there are some programs that won't let students 
out until they have answered five or ten or fifteen questions 
correctly in a row. Some students just can't do that, and they 
get so frustrated by a lack of success that they quit. Keep 
this in mind as you evaluate a progran:i. In addition, remember 
to check whether the program keeps track of the student's 
work, giving some type of summary and evaluation at the end. 



What Pedagogical Issues Should I Consider 
During the First Viewing? 

There are several important issues. For example, is the se- 
quencing pedagogically logical? Does the program permit the 
student to interact-think, respond, wonder, predict-or does 
the student just read an electronic workbook? (You can buy 
quite a few workbooks for the cost of one piece of software, 
not to mention the cost of the computer.) The more inter- 
active the program, the better. Get students involved in this 
evaluation and make sure that computer involvement is worth- 
while for both their education and their time. 

Look carefully at the language: Is it too formal, or too 
laden with slang? Is it so dependent on fad that it will be 
outdated in two years? Finally, do graphics and/or sound add 
anything to the program, or are they window dressing? 

What Should I Look for in a Second Viewing? 

If the program seems to pass the test so far~you like the 
objectives, it's user-friendly, and tht pedagogy is good-ii's 
time for a second round of assessment, in which you con- 
sciously make errors. This may seem hard. Teachers like to do 
things correctly and neatly. On this run-through, however, 
you'll need to force yourself to be a confused and/or slow 
learner. 

Find out what happens if you just hit the return key with 
no input. Can you do it three times, get the correct answer, 
and page on through or do you have to make an attempt to 
answer? What does your program do with perceptually correct 
but literally incorrect answers-especially misspellings, parts 
of names, or abbreviations? What does the program do with 
a totally "off-the-wall" response^treat it as such or say, "Wow, 
that's interesting, Johnnie"? What happens if you hit the 
escape key, hit many keys simultaneously, or randomly tap 
the keyboard? Try it. If the program is capable of handling 
such things, it's said to be "bulletproof." If the program isn't 
designed to handle accidental or intentional problems, it's 
going to backfire on some students. 

If you look at these aspects, you will have given the pro- 
gram a good preview. If you still like the program at this 
point, then it's time to turn the program over to some of 
your students. Choose at least a couple-the quick/bright and 
the siow/unmotivated-and get their opinions. Their actions 
*and reactions will reveal much more about the program. 

What Should I Know about Dealing 
with Software Publishers? 

If publishers will not let you inspect programs on atrial basis, 
beware but remember that they are not entirely at fault. 
Software development, especially well-done material, costs 
money. If publishers send out disks for preview and you copy 
them and send them back, they stand to lose a great deal of 



money. So allow for the publisher's qualms when making 
arrangements for previewing. 'Send a letter guaranteeing that 
you will not copy iljegally. In addition, request to see the real 
program rather than shortened preview disks. If they won't 
send just the disks, ask them to send a sales representative 
to do a demonstration and then try to get time for your own 
evaluation. Sometimes several schools can join together and 
invite several publishers to put on a software fair with a wider 
rangiB of materials. It takes planning, but it is a way to see 
the material. 

Another factor to check on is how to get a backup disk 
or multiple copies. You should be given either a backup disk 
or provision to make your own. Also, how does the publisher 
deal with your need for five, ten, fifteen or thirty copies for 
your district? Paying the full retail price for each one could 
wipe out a budget for software on one item. Negotiate a 
reasonable, nominal fee for additional oi replacement copies. 
In school settings, disks are sometimes lost, stepped on, or 
otherwise damaged. In addition, you should investigate the 
publisher's policy if, for some reason you didn't foresee, the 
program just will not work out. Do not, however, make the 
return option an excuse for weak evaluation practice on your 
part. 

Finally, deal openly and honestly with publishers and 
expect to be treated in the same manner. Remember, they are 
somewhat new at this also. The copyright laws are ambiguous 
on computer materials and business policies and practices 
are just being formed. Publishers and educators must work 
together to establish a smooth working relationship. Tell 
your colleagues and educators in other districts about pub- 
lishers who are good to work with. Set up a network of 
information. 

In addition, make use of general professional resources, 
like reviews of software from EPIE (Educational Products 
Information Exchange) and MicroSift. Read software reviews 
and gather ideas about the instructional use of the computer 
in journals like Language Arts, English Journal, and Computers 
in Reading and Language Arts. But don't let others' ideas 
take the place of your own hands-on assessment. The ideas 
in this digest, along with documents like /'Guidelines for 
Review and Evaluation of English Language Arts Software' 
(National Council of Teachers of English 1984) can help you 
to choose the best software for use with your students. 

Pamela Janello 
University of Illinois 
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